System and method for discerning complex gestures using an array of optical sensors

ABSTRACT

A method for gesture determination (e.g., discerning complex gestures) via an electronic system (e.g., a gesture sensing system) including an array of optical sensors is described herein. The method includes detecting a plurality of sub-gestures (e.g., simple gestures provided by a target located proximate to the system) via the array of optical sensors. The sensors generate signals based upon the detected (e.g., received) sub-gestures and transmit the signals to a processor of the gesture sensing system. The processor processes the signals to obtain data associated with the sub-gestures and analyzes the sub-gesture data to determine if the sub-gestures collectively constitute a gesture (e.g., complex gesture). When the analyzing indicates that the sub-gestures collectively constitute a complex gesture, the gesture sensing system detects the complex gesture.

BACKGROUND

A gesture sensor is a human interface device that enables the detectionof physical movement by a user without the user actually touching thedevice within which the gesture sensor resides. The detected movementscan be subsequently used as input commands for the device. In someapplications, the device is programmed to recognize distinct non-contacthand motions, such as left-to-right, right-to-left, up-to-down,down-to-up, in-to-out and out-to-in hand motions. Gesture sensors havefound popular use in handheld devices, such as tablet computing devicesand smartphones, and other portable devices, such as laptops. Gesturesensors are also being implemented in video game consoles that detectthe motion of the video game player.

SUMMARY

A method for gesture determination (e.g., discerning complex gestures)via an electronic system (e.g., a gesture sensing system) including anarray of optical sensors is described herein. The method includesdetecting a plurality of sub-gestures (e.g., simple gestures provided bya target located proximate to the system) via the array of opticalsensors. The sensors generate signals based upon the detected (e.g.,received) sub-gestures and transmit the signals to a processor of thegesture sensing system. The processor processes the signals to obtaindata associated with the sub-gestures and analyzes the sub-gesture datato determine if the sub-gestures collectively constitute a gesture(e.g., complex gesture). When the analyzing indicates that thesub-gestures collectively constitute a complex gesture, the gesturesensing system detects the complex gesture.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

DRAWINGS

The detailed description is described with reference to the accompanyingfigures. The use of the same reference numbers in different instances inthe description and the figures may indicate similar or identical items.

FIG. 1 is a conceptual block diagram of an optical gesture sensor inaccordance with an exemplary embodiment of the present disclosure.

FIG. 2 is a conceptual block diagram of a gesture sensing system inaccordance with an exemplary embodiment of the present disclosure.

FIG. 3 is an exemplary flow diagram for a sub-gesture collection andsub-gesture data processing sequence in accordance with an exemplaryembodiment of the present disclosure.

FIGS. 4A-4D depict example lookup tables which may be implemented by thegesture sensing system when analyzing sub-gesture data in accordancewith exemplary embodiments of the present disclosure.

FIG. 4E depicts an example lookup table which may be implemented by thegesture sensing system, the lookup table providing possible combinationsof sub-gestures which can be received by the sensors, and whether or notthe combinations would result in the combination of sub-gestures beinginterpreted as a complex gesture (e.g., whether or not a complex gesturewould be detected) by the gesture sensing system, in accordance with anexemplary embodiment of the present disclosure.

FIG. 5A depicts an example sequence tree which may be implemented by thegesture sensing system when analyzing sub-gesture data in accordancewith an exemplary embodiment of the present disclosure.

FIG. 5B depicts valid combinations of sub-gestures which, if received bythe gesture sensing system, may be interpreted as complex gestures(e.g., pinch gesture or a stretch gesture) in accordance with anexemplary embodiment of the present disclosure.

FIGS. 6A and 6B depict a flow diagram illustrating an example processfor gesture determination via a gesture sensing system in accordancewith an exemplary embodiment of the present disclosure.

DETAILED DESCRIPTION Overview

Gesture sensing systems enable the detection of physical movement by auser without the user actually touching the gesture sensing system. Todetect the physical movements (e.g., gestures), the gesture sensingsystem implements an optical sensor (e.g., gesture sensor) which isintegrated with the gesture sensing system. The optical sensorfacilitates the ability of the gesture sensing system to recognizesimple gestures, such as left-to-right, right-to-left, up-to-down,down-to-up, in-to-out and out-to-in hand motions.

Described herein is a gesture sensing system (and a method of operationof the gesture sensing system) which is configured for detecting complexgestures (e.g., a stretch gesture, a pinch gesture) utilizing an arrayof optical sensors, each of which individually can only recognize simplegestures. A framework for processing and analyzing the collective dataobtained via the array of optical sensors is implemented by the gesturesensing system, as described herein, to discern the complex gesturesbased upon the plurality of simple gestures obtained via the array ofsensors.

Example Implementations

FIG. 1 is a conceptual block diagram of a gesture sensing device (e.g.,a sensor, a gesture sensor, an optical sensor, an optical gesturesensor) in accordance with an exemplary embodiment of the presentdisclosure. In embodiments, the gesture sensing device 100 includes asingle illumination source, represented as light-emitting diode (LED)102. In embodiments, the gesture sensing device 100 includes a photosensor (e.g., a segmented photo sensor) 104. In some embodiments, thephoto sensor 104 is configured for sensing (e.g., detecting) only aspecific wavelength or wavelengths of light, such as the wavelengthsemitted by the illumination source 102. Such a configuration can beimplemented through the use of a filter. In embodiments, the photosensor 104 can be either a single sensor functionally partitioned intomultiple segments or it can be an array of individual photo sensors. Forexample, a quad segmented photo sensor is functionally equivalent tofour individual photo sensors arranged in a quad layout. As used herein,reference to a “segment” refers to either a partitioned segment within asingle sensor or to an individual sensor in an array of sensors. FIG. 1shows the photo sensor 104 as both on-edge (upper element labeled 104)and a plan view to show the different segments (lower element labeled104).

In the exemplary configuration of FIG. 1, the photo sensor 104 includesfour segments: segment A, segment B, segment C and segment D. Although afour-segment sensor (e.g., detector) is one implementation, it isunderstood that the number of segments can be increased to increase theresolution of the device 100. The signal processing electronics becomeincreasingly more complex as the number of segments is increased. Inembodiments, each of the segments of the photo sensor 104 are isolatedfrom each other. In embodiments, the illumination source (e.g., LED) 102is positioned (e.g., located) proximate to the segmented photo sensor104. When a moving target passes proximate to the illumination source102 and the photo sensor 104, light output from the illumination source102 is reflected off the moving target and to the photo sensor 104. Inembodiments, the gesture sensing device 100 includes an optical lensstructure 106 configured for focusing light onto the photo sensor 104.In embodiments, the optical lens structure (e.g., focusing lens) 106focuses reflected light from a moving target, such as a hand making agesture, in the space above the photo sensor 104. In embodiments, onlylight that is within the field of view of the optical lens structure 106is focused onto the photo sensor 104. Although shown as a single elementin FIG. 1, the optical lens structure 106 may include any number oflenses and/or optical elements for directing light to the photo sensor104. In embodiments, each segment of the photo sensor 104 outputs asegment signal to a control circuit 108 which is connected to the photosensor 104. In embodiments, the control circuit 108 is configured forprocessing the segment signals received from the photo sensor 104.

In embodiments, the illumination source 102 is configured for beingcontinuously or periodically energized (e.g., powered on) forilluminating the target. In embodiments, the light reflected from thetarget induces the segment signals on the photo sensor(s) 104. Inembodiments, these segment signals are received and processed by thecontrol circuit 108. In embodiments, the processed segment signals arestored in a buffer memory, the buffer memory being integrated with orseparate from the control circuit 108. In embodiments, the controlcircuit 108 is configured for analyzing the stored signals (e.g., storeddata) and determining if a valid gesture has been detected. Inembodiments, the stored data may also be used so that the photo sensor104 operates as a proximity detector. In embodiments, the same photosensor 104 can be used with a different signal processing circuit sothat the gesture sensing device 100 also functions as an ambient lightsensor.

In embodiments, when the illumination source 102 is powered on, orflashes, the target is illuminated if the target is within a proximatespace above the photo sensor 104. The moving target is conceptuallyillustrated in FIG. 1 as a flat reflector. The target reflection isimaged by the optical lens structure 106 onto the photo sensor 104. Theexample of FIG. 1 illustrates a right-to-left motion of the target. Asthe edge of the target moves through the center of the imaging zone, thefocused image of the target edge moves across the photo sensor 104.Segments A and C respond first to the moving image, followed by segmentsB and D. In embodiments, the control circuit 108 is programmed to detectthis sequence of events, and recognizes a right-to-left target motion.Similarly, a left-to-right target motion can be recognized, via theopposite sequence, and both up-to-down and down-to-up target motions canbe recognized using the orthogonal set of signals. In-and-out targetmotion can be recognized by the control circuit 108 via sensing of theabsolute amplitude of the sum of the four segments (A through D), whichis also the proximity measurement. In some embodiments, filters areimplemented on top of the photo sensor(s) 104 to filter out light havingwavelengths that are different than the illumination source 102.

FIG. 2 is a conceptual block diagram of a gesture sensing system 200 inaccordance with an exemplary embodiment of the present disclosure. Inembodiments, the system 200 is an electronic system (e.g., an electronicdevice). For instance, the system 200 can be a tablet computing device,a smartphone, a cell phone, a notebook computing device, a laptopcomputing device, or a video game console.

In embodiments, the gesture sensing system 200, is a multi-sensor systemwhich implements a plurality (e.g., an array) of sensors (e.g., opticalsensors, gesture sensors) 202. In embodiments, each sensor 202implemented in the system 200 can be the single-LED optical gesturesensor (e.g., gesture sensor, gesture sensing device) 100 describedabove, and further described in the co-owned and co-pending U.S. PatentApplication Pub. No. US 2012/0280904, filed Jan. 17, 2012, and entitled“Method For Detecting Gestures Using a Multi-Segment Photodiode and Oneor Fewer Illumination Sources”, and the co-owned and co-pending U.S.Patent Application Pub. No. 2012/0280107, filed Nov. 25, 2011, andentitled “Optical Gesture Sensor Using a Single Illumination Source”which are herein incorporated in their entireties by reference. Infurther embodiments, the sensors 202 implemented in the system 200 canbe any one of a number of other types of optical sensors (e.g., gesturesensors, gesture sensing devices), such as any one of the other gesturesensing devices described in the co-owned and co-pending U.S. PatentApplication Pub. No. US 2012/0280904, filed Jan. 17, 2012, and entitled“Method For Detecting Gestures Using a Multi-Segment Photodiode and Oneor Fewer Illumination Sources”, and the co-owned and co-pending U.S.Patent Application Pub. No. 2012/0280107, filed Nov. 25, 2011, andentitled “Optical Gesture Sensor Using a Single Illumination Source”. Infurther embodiments, the sensors 202 implemented in the system 200described herein can be any gesture sensing device (e.g., gesturesensor, optical sensor).

In the illustrated embodiment, the plurality of sensors 202 implementedby the system 200 includes four sensors 202. In alternative embodiments,the plurality of sensors 202 may include a different number of (e.g.,more than four, fewer than four) sensors 202. In embodiments, eachsensor 202 included in the plurality of sensors 202 is located (e.g.oriented) at a fixed location within or on the system (e.g., device)200. For example, the sensors 202 may be placed at fixed locationsproximate to a display screen of the system (e.g., device) 200. Inembodiments, it is contemplated that the number of sensors 202implemented by the system 200 can be directly proportional to the numberof gestures the system 200 is able to detect, so more sensors 202 may beused for expanding the gesture detection capabilities of the system 200.

In embodiments, the system 200 further includes a processor (e.g., acentral processor) 204 which is connected to each sensor included in theplurality of sensors 202 of the system 200. The processor 204 providesprocessing functionality for the system 200 and can include any numberof processors, micro-controllers, or other processing systems andresident or external memory for storing data and other informationaccessed or generated by the system 200. The processor 204 can executeone or more software programs that implement techniques describedherein. The processor 204 is not limited by the materials from which itis formed or the processing mechanisms employed therein and, as such,can be implemented via semiconductor(s) and/or transistors (e.g., usingelectronic integrated circuit (IC) components), and so forth.

In embodiments, the sensors 202 are configured for sensing (e.g.,detecting) gestures made by the target (e.g., a moving hand). Forexample, each sensor 202 is configured for sensing (e.g., detecting)simple gestures (e.g., simple movements, simple three-dimensional (3-D)gestures, sub-gestures), which can include, but are not limited to: aleft-to-right gesture (e.g., a left-to-right hand swipe), aright-to-left gesture, an up-to-down gesture, a down-to-up gesture, anin-to-out gesture, an out-to-in gesture and a clicking gesture. Inembodiments, the gestures are touch-free in that the gestures are sensed(e.g., detected) by the sensors 202 without the target (e.g., the sourceof the gestures) physically contacting the sensors 202. In embodiments,the sensors 202 are further configured for generating signals based uponthe sensed (e.g., detected) gestures. In embodiments, the sensors 202are further configured for transmitting the signals to the processor204.

In embodiments, the processor 204 is configured for receiving thesignals transmitted by the sensors 202. In embodiments, the processor204 is configured for processing the signals received from the sensors202 to derive gesture data from the signals. In embodiments, theprocessor 204 is configured for collectively processing and analyzingthe signals (and the gesture data) corresponding to sub-gesturesreceived (e.g., detected by) the sensors 202 of the array to determinewhether a complex gesture has been sensed (e.g., detected, received) bythe array of sensors 202 and, if so, what type of complex gesture hasbeen sensed by the array of sensors 202. In embodiments, a complexgesture (e.g., gesture) is a gesture derived from a combination ofsimple gestures (e.g., sub-gestures). For example, complex gestures caninclude, but are not limited to, an erase gesture, a pinch gesture, astretch gesture, a clockwise rotate gesture, and a counterclockwiserotate gesture. In embodiments, a complex gesture can have a meaningwhich is different from the meanings of the individual simple gesturesfrom which the complex gesture is derived. For example, a complexgesture, such as an erase gesture, can be derived from a combination ofindividual simple gestures, such as a left-to-right gesture, followed bya right-to-left gesture.

In embodiments, processing functionality of the processor 204 can beimplemented via a software program running (e.g., executing) on theprocessor 204. In embodiments, the software program can implement one ormore algorithms. In embodiments, the system 200 includes memory 206(e.g., digital memory). In embodiments, the memory 206 is connected to(e.g., accessible by) the processor 204 and is configured for storingdata. The memory 206 is an example of tangible computer-readable storagemedium that provides storage functionality to store various dataassociated with operation of the system 200, such as software programsand/or code segments, or other data to instruct the processor 204 toperform the steps described herein. Although a single memory 206 isshown, a wide variety of types and combinations of memory can beemployed. The memory 206 can be integral with the processor 204, cancomprise stand-alone memory, or can be a combination of both. The memory206 can include, but is not necessarily limited to: removable andnon-removable memory components, such as random-access memory (RAM),read-only memory (ROM), flash memory (e.g., a secure digital (SD) memorycard, a mini-SD memory card, and/or a micro-SD memory card), magneticmemory, optical memory, universal serial bus (USB) memory devices, andso forth. In embodiments, the memory 206 can include removableintegrated circuit card (ICC) memory, such as memory provided by asubscriber identity module (SIM) card, a universal subscriber identitymodule (USIM) card, a universal integrated circuit card (UICC), and soon.

In embodiments, the memory 206 is configured for storing data whichprovides information about the relative locations of all sensors 202 ofthe system 200. In embodiments, the memory 206 is configured for storingone or more lookup tables and/or sequence trees. For example, the lookuptables and/or sequence trees can include pre-determined potentialcombinations and/or sequences of sub-gestures (e.g., simple gestures)which may be detected by the sensors 202 of the system 200 and thecorresponding gestures (e.g., complex gestures), if any, associated with(e.g., defined by or indicated by) those respective combinations and/orsequences. The processor 204 is configured for utilizing (e.g.,referencing) the lookup tables and/or sequence trees for promoting theability of the system 200 to: i) determine if a complex gesture has beendetected by the system 200 and; ii) if so, define the type of complexgesture detected by the system 200.

As mentioned above, the processor 204 is configured for collectivelyprocessing and analyzing the signals (and the gesture data)corresponding to sub-gestures received (e.g., detected) by the sensors202 of the array to determine whether a complex gesture has been sensed(e.g., detected, received) by the array of sensors 202 and, if so, whattype of complex gesture has been sensed by the array of sensors 202. Thesystem 200 is configured for carrying out this processing in any one ormore of a variety of ways as will be discussed below.

In some embodiments, the system 200 is configured for starting acountdown timer when a sub-gesture (e.g., simple gesture) is received on(e.g., detected by) any of the sensors (e.g., sensors 1-4) 202 of thesystem 200. For example, the duration of the countdown timer can be lessthan 500 milliseconds. The starting of the countdown timer commences adata frame for the system 200, which ends when the countdown timerexpires. During the data frame, sub-gestures (e.g., simple gestures) canbe received (e.g., detected) by the sensors 202. After the data frameexpires, the processor 204 then collectively processes and analyzes thesignals (and the gesture data) corresponding to the sub-gestures whichwere received (e.g., detected) by the sensors 202 of the array duringthe data frame to determine whether a complex gesture has been sensed(e.g., detected, received) by the array of sensors 202 and, if so, whattype of complex gesture has been sensed by the array of sensors 202.FIG. 3 (FIG. 3) illustrates an exemplary flow diagram for theabove-described gesture (e.g., sub-gesture) collection and gesture dataprocessing sequence.

As mentioned above, when processing and analyzing the gesture data(e.g., sub-gesture data) to determine whether a complex gesture has beensensed (e.g., detected) by the array of sensors 202, the processor 204can access a lookup table. FIGS. 4A-4D (FIGS. 4A-4D) illustrate examplesof lookup tables which may be implemented by the system 200 shown inFIG. 2. FIG. 4A depicts a table which provides a series of sub-gesturecombinations which, if received by the sensors specified in the table,would define a complex gesture (e.g., a stretch gesture). For example,the first entry in the table of FIG. 4A indicates that if no simplegestures (e.g., sub-gestures) were received by sensors 1 and 2, adown-to-up sub-gesture was received by sensor 3, and a left-to-rightsub-gesture was received by sensor 4, such a sequence of sub-gestureswould be collectively interpreted (e.g., detected) as a complex gesture(e.g., a stretch gesture). Further, the last entry in the table of FIG.4A indicates that if a right-to-left sub-gesture was received by sensor1, no sub-gesture was received by sensor 2, and a left-to-rightsub-gesture was received by sensor 4, regardless of what sub-gesture wasreceived by sensor 3, the sequence of sub-gestures would be collectivelyinterpreted (e.g., detected) as a stretch gesture. FIGS. 4B through 4Ddepict tables which provide sub-gesture combinations which, if receivedby the sensors specified in the tables, would define other complexgestures including a pinch gesture (FIG. 4B), a rotate clockwise gesture(FIG. 4C), and a rotate counterclockwise gesture (FIG. 4D). Referring tothe first entry in the table depicted in FIG. 4B, if no sub-gestures arereceived (e.g., detected) by sensors 1 and 2, a left-to-rightsub-gesture is received by sensor 3 and a right-to-left sub-gesture isreceived by sensor 4, the sequence of sub-gestures would be collectivelyinterpreted (e.g., detected) as a complex gesture (e.g., a pinchgesture). For reference purposes, the arrows in the tables shown inFIGS. 4A-4D indicate the direction of motion of the sub-gesture receivedby that sensor, while an “X” indicates an inconsequential sub-gesture,such that it doesn't matter what sub-gesture is received by that sensor,the combination will still be collectively interpreted as a complexgesture. Further, in the tables shown in FIGS. 4A-4D the quotation markindicates that no gesture was received by that sensor. For example, inthe second entry of FIG. 4A, no gesture was received by sensors 1 and 2.

FIG. 4E depicts another lookup table which provides possiblecombinations of sub-gestures (e.g., simple gestures) which can bereceived by the sensors, and whether or not the combinations wouldresult in the combination of simple gestures being interpreted as acomplex gesture (e.g., whether or not a complex gesture would bedetected) by the system 200 shown in FIG. 2. For example, the secondentry in the table of FIG. 4E depicts a combination of receivedsub-gestures in which no sub-gesture was received by sensor 1, anup-to-down sub-gesture was received by sensor 2, a left-to-rightsub-gesture was received by sensor 3 and a right-to-left sub-gesture wasreceived by sensor 4. The combination of sub-gestures received bysensors 3 and 4 would normally be interpreted as a pinch gesture.However, because there is another (e.g., an extra) sub-gesture present(e.g., the sub-gesture received by sensor 2) which, when combined withthe other received sub-gestures, does not fit (e.g., match) apre-determined combination of sub-gestures that defines a complexgesture, the system 200 interprets (e.g., treats) this as no gesturebeing detected by the system 200.

As mentioned above, after the data frame expires, the processor 204collectively processes and analyzes the signals (and the gesture data)corresponding to the sub-gestures which were received (e.g., detected)by the sensors 202 of the array during the data frame to determinewhether a complex gesture has been sensed (e.g., detected, received) bythe array of sensors 202 and, if so, what type of complex gesture hasbeen sensed by the array of sensors 202. In embodiments, if just theinitial isolated sub-gesture (e.g., the sub-gesture that triggered thecountdown timer) was received, and no other sub-gestures were receivedby any of the sensors 202 within the data frame, the system 200 wouldinterpret (e.g., treat) this as no gesture being detected by the system200. Examples of this scenario are depicted in the fourth and fifthentries of the table shown in FIG. 4E.

In other embodiments, when processing and analyzing the gesture data(e.g., sub-gesture data) to determine whether a complex gesture has beensensed (e.g., detected) by the array of sensors 202, the processor 204can access a sequence tree, as mentioned above. The sequence tree caninclude pre-determined sequences of sub-gestures (e.g., simple gestures)and the corresponding gestures (e.g., complex gestures) associated with(e.g., defined by) those sub-gesture sequences. For the case where thereare N types of simple gestures (e.g., sub-gestures) available, there canbe N sequence trees defining gesture sequences that can be traversed.These N sequence trees can be a set of defined trees which are traversedbased upon the sub-gestures received by the system 200. For example, thefirst node of the sequence tree can correspond to the first sub-gesturereceived, with each subsequent node of the sequence tree being aspecific gesture (e.g., a simple gesture or a complex gesture) which canbe sensed (e.g., detected) by one or more of the sensors 202. Inembodiments, when a goal node of the tree is reached (e.g., a nodecorresponding to a last sub-gesture in a valid, pre-defined sequence ofsub-gestures defining a complex gesture, the system 200 detects thecomplex gesture. In embodiments, multiple sequence trees can be mergedinto a single tree for the sake of efficiency. The strength of thesequence tree approach is that it is based upon the activity-start,activity-stop paradigm and is also recursive, meaning that it ispossible to stop and re-start sequences easily.

FIG. 5A depicts an example of a sequence tree, such as described above.For example, assuming the system 200 shown in FIG. 2 is being utilized,FIG. 5A illustrates that if a left-to-right sub-gesture is received bysensor 3 (sub-gesture “3R”) followed by a right-to-left sub-gesturebeing received by sensor 4 (sub-gesture “4L”), then the combination isinterpreted by the system 200 as a complex gesture (e.g., a pinchgesture). Further, FIG. 5A illustrates that if a left-to-rightsub-gesture is received by sensor 3 (sub-gesture “3R”), followed by aleft-to-right sub-gesture being received by sensor 4 (sub-gesture “4R”),then the combination is interpreted as a sweep right gesture. Further,FIG. 5A illustrates that if a left-to-right sub-gesture is received bysensor 3 (sub-gesture “3R”), followed by a left-to-right sub-gesturebeing received by sensor 4 (sub-gesture “4R”), followed by a down-to-upsub-gesture being received by sensor 2 (sub-gesture “2U”), then thecombination is interpreted as an L gesture. Further, FIG. 5A illustratesthat if a left-to-right sub-gesture is received by sensor 4 (sub-gesture“4R”), followed by a right-to-left sub-gesture being received by sensor3 (sub-gesture “3L”), then the combination is interpreted as a stretchgesture. Still further, FIG. 5A illustrates how some combinations ofsub-gestures received by the system 200 may be determined as notdefining complex gestures. For example, if a left-to-right sub-gestureis received by sensor 4 (sub-gesture “4R”) and a down-to-up sub-gestureis received by sensor 2 (sub-gesture “2U”), this combination ofsub-gestures is interpreted by the system 200 as not defining a complexgesture, and so the system 200 treats this combination as though nogesture was detected by the system 200.

In some embodiments, the system 200 can implement a hybrid approach inwhich the sequence tree is used in combination with the countdown timer.This hybrid approach combines the start-stop activity paradigm with thecountdown timer to allow timeouts to cease detection and forceprocessing of data after a given amount of time has expired. In suchembodiments, if just the initial isolated sub-gesture (e.g., thesub-gesture that triggered the countdown timer) was received, and noother sub-gestures were received by any of the sensors 202 within thedata frame, the system 200 would interpret this as no gesture beingdetected by the system 200. However, if at the end of a data frame, alast sub-gesture included in the sequence of sub-gestures receivedwithin the data frame corresponds to a terminating node of the sequencetree, this is an indication that an entire, correct sequence ofsub-gestures corresponding to a complex gesture have been received andthe system 200 will detect the complex gesture. FIG. 5B (FIG. 5B)illustrates valid combinations of sub-gestures which, if received, willbe interpreted as complex gestures, such as a pinch gesture or a stretchgesture.

Example Complex Gesture Determination Processes

FIGS. 6A and 6B (FIGS. 6A and 6B) depict a flowchart illustrating anexample process (e.g., method) for gesture determination via a system(e.g., electronic device) in accordance with an exemplary embodiment ofthe present disclosure. In embodiments, the method 600 is performed by asystem (e.g., electronic device, gesture sensing system), such as thesystem 200 described above. In embodiments, the method 600 includes thestep of detecting (e.g., sensing) a plurality of sub-gestures (e.g.,simple gestures) via a plurality of gesture sensors 202 of the system(Step 602). For example, the plurality of sub-gestures is provided by amoving target located proximate to the electronic system 200.

In embodiments, the method 600 further includes the step of generating aplurality of signals via the plurality of sensors 202, the plurality ofsignals being based on the plurality of sub-gestures (Step 604). Inembodiments, the method 600 further includes the step of transmittingthe plurality of signals via the plurality of sensors 202 to a processor204 of the system (Step 606). In embodiments, the method 600 furtherincludes the step of processing the plurality of signals via theprocessor 204 to obtain sub-gesture data associated with the pluralityof sub-gestures (Step 608). For example, the processor 204 of the system200 processes the signals to determine which simple gestures werereceived (e.g., detected) by the sensors 202.

In embodiments, the method 600 further includes the step of analyzingthe sub-gesture data associated with the plurality of sub-gestures viathe processor 204 to determine if the plurality of sub-gestures (e.g.,simple gestures) collectively constitute a gesture (e.g., complexgesture) (Step 610). For example, the processor 204 analyzes theplurality of simple gestures and determines if they collectivelyconstitute a complex gesture. In some embodiments, the analyzing stepcan include a step of comparing the sub-gesture data to a lookup tableand when the sub-gesture data corresponds to a pre-determinedsub-gesture combination in the table which is associated with a complexgesture, classifying the plurality of received sub-gestures as thecomplex gesture (Step 610 a). For example, the lookup table can includea plurality of possible pre-determined sub-gesture combinationscorresponding to gestures (e.g., complex gestures). In otherembodiments, the analyzing step can include a step of comparing thesub-gesture data to a sequence tree and when the sub-gesture datacorresponds to a pre-determined sub-gesture sequence of the sequencetree which is associated with a complex gesture, classifying theplurality of received sub-gestures as the complex gesture (Step 610 b).For example, the sequence tree can indicate a plurality of possiblepre-determined sub-gesture sequences corresponding to gestures (e.g.,complex gestures).

In embodiments, when analyzing the sub-gesture data associated with theplurality of sub-gestures indicates that the plurality of sub-gesturescollectively constitute a gesture, interpreting the plurality ofsub-gestures as a detected gesture (Step 612). For example, whenanalysis of the sub-gesture data indicates that a combination and/orsequence of sub-gestures has been received which is associated with(e.g., defines) a complex gesture, the system 200 detects the complexgesture.

CONCLUSION

Although the subject matter has been described in language specific tostructural features and/or process operations, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

What is claimed is:
 1. A method for gesture determination via anelectronic system, the method comprising: detecting a plurality ofsub-gestures via a plurality of sensors of the electronic system; when afirst sub-gesture included in the plurality of sub-gestures is detected,commencing a data frame for the system, the data frame having apre-determined, finite duration; generating a plurality of signals viathe plurality of sensors, the plurality of signals being based on theplurality of sub-gestures; transmitting the plurality of signals via theplurality of sensors to a processor of the system; processing theplurality of signals via the processor to obtain sub-gesture dataassociated with the plurality of sub-gestures; and analyzing thesub-gesture data associated with the plurality of sub-gestures via theprocessor to determine if the plurality of sub-gestures collectivelyconstitute a gesture, wherein analyzing the sub-gesture data isperformed after the data frame expires and is limited to analyzing thesub-gesture data corresponding to the first sub-gesture and tosub-gestures detected during the data frame, and wherein the pluralityof sub-gestures are provided by a moving target located proximate to theelectronic system.
 2. The method as claimed in claim 1, furthercomprising: when analyzing the sub-gesture data associated with theplurality of sub-gestures indicates that the plurality of sub-gesturescollectively constitute a gesture, interpreting the plurality ofsub-gestures as a detected gesture.
 3. The method as claimed in claim 1,wherein commencing the data frame for the system includes starting acountdown timer.
 4. The method as claimed in claim 2, wherein analyzingthe sub-gesture data includes: comparing the sub-gesture data to alookup table, the lookup table including a pre-determined sub-gesturecombination corresponding to the gesture.
 5. The method as claimed inclaim 2, wherein analyzing the sub-gesture data associated with theplurality of sub-gestures includes: comparing the sub-gesture data to asequence tree, the sequence tree indicating a pre-determined sub-gesturesequence corresponding to the gesture.
 6. The method as claimed in claim4, further comprising: when the sub-gesture data corresponds to thepre-determined sub-gesture combination in the table, classifying theplurality of received sub-gestures as the gesture.
 7. The method asclaimed in claim 5, further comprising: when the sub-gesture dataindicates that the plurality of received sub-gestures corresponds to thepre-determined sub-gesture sequence, classifying the plurality ofreceived sub-gestures as the gesture.
 8. A computer-implemented methodfor gesture determination, the computer-implemented method comprising:detecting a plurality of sub-gestures; when a first sub-gesture includedin the plurality of sub-gestures is detected, commencing a data frame,the data frame having a pre-determined, finite duration; generating aplurality of signals based on the plurality of detected sub-gestures;causing a processor to process the plurality of signals to obtainsub-gesture data associated with the plurality of sub-gestures; causingthe processor to analyze the sub-gesture data associated with theplurality of sub-gestures to determine if the plurality of sub-gesturescollectively constitute a gesture, wherein analyzing the sub-gesturedata is performed after the data frame expires and is limited toanalyzing the sub-gesture data corresponding to the first sub-gestureand to sub-gestures detected during the data frame; and when analyzingthe sub-gesture data associated with the plurality of sub-gesturesindicates that the plurality of sub-gestures collectively constitute thegesture, detecting the gesture, wherein the plurality of sub-gesturesare provided by a moving target located proximate to the electronicsystem.
 9. The computer-implemented method as claimed in claim 8,wherein the sub-gestures have a first level of complexity and thegesture has a second level of complexity which is greater than the firstlevel of complexity.
 10. The computer-implemented method as claimed inclaim 8, wherein commencing the data frame includes starting a countdowntimer.
 11. The computer-implemented method as claimed in claim 8,wherein analyzing the sub-gesture data includes: comparing thesub-gesture data to a lookup table, the lookup table including apre-determined sub-gesture combination corresponding to the gesture. 12.The computer-implemented method as claimed in claim 8, wherein analyzingthe sub-gesture data associated with the plurality of sub-gesturesincludes: comparing the sub-gesture data to a sequence tree, thesequence tree indicating a pre-determined sub-gesture sequencecorresponding to the gesture.
 13. The computer-implemented method asclaimed in claim 12, further comprising: when the sub-gesture datacorresponds to the pre-determined sub-gesture combination in the table,classifying the plurality of received sub-gestures as the gesture. 14.The computer-implemented method as claimed in claim 13, furthercomprising: when the sub-gesture data indicates that the plurality ofreceived sub-gestures corresponds to the pre-determined sub-gesturesequence, classifying the plurality of received sub-gestures as thegesture.
 15. An electronic system, comprising: a plurality of sensors,the plurality of sensors configured for detecting a plurality ofsub-gestures provided by a moving target located proximate to thesystem, the plurality of sensors configured for generating andtransmitting a plurality of signals based on the plurality ofsub-gestures; a processor, the processor being connected to theplurality of sensors and being configured for receiving the plurality ofsignals transmitted by the plurality of sensors; and a memory, thememory being communicatively coupled with the processor, the memoryhaving computer executable instructions stored thereon, the computerexecutable instructions configured for execution by the processor to:process the plurality of signals to obtain sub-gesture data associatedwith the plurality of sub-gestures; when a first sub-gesture included inthe plurality of sub-gestures is detected, commence a data frame, thedata frame having a predetermined, finite duration; analyze thesub-gesture data associated with the plurality of sub-gestures todetermine if the plurality of sub-gestures collectively constitute agesture, wherein the sub-gesture data is analyzed after the data frameexpires and is limited to analysis of the sub-gesture data correspondingto the first sub-gesture and to sub-gestures detected during the dataframe; and detecting the gesture when analysis of the sub-gesture dataindicates that plurality of sub-gestures collectively constitute thegesture.
 16. The electronic system as claimed in claim 15, wherein theplurality of sensors are optical sensors.